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A wide range of occupations require science, technology, engineering, and mathematics (STEM) skills, yet 
almost half of students who intend to pursue a postsecondary STEM education abandon these plans before 
graduating from college. This attrition is especially pronounced among underrepresented groups (i.e., racial/ 
ethnic minorities and first-generation college students). We conducted a 2-year follow-up of a utility-value 
intervention that had been implemented in an introductory biology course. This intervention was previously 
shown to improve performance in the course, on average and especially among underrepresented students, 
reducing the achievement gap. The goal of the present study was to examine whether the intervention also 
impacted persistence in the biomedical track throughout college. The intervention had a more positive impact 
on long-term persistence for students who were more confident that they could succeed at the beginning of 
the course, and this effect was partially driven by the extent to which students reflected on the personal 
relevance of biological topics in their essays. This mechanism was distinct from the process that had been 
found to underlie intervention effects on performance—engagement with course material—suggesting that 
utility-value interventions may affect different academic outcomes by initiating distinct psychological pro- 
cesses. Although we did not find that the intervention was differentially effective for underrepresented students 
in terms of persistence, we found that positive effects on performance were associated with increased 
persistence for these students. Results suggest that utility-value interventions in an introductory course can be 
an effective strategy to promote persistence in the biomedical sciences throughout college. 


Educational Impact and Implications Statement 

We conducted a 2-year follow-up study of a utility-value intervention (UVJ) in which students wrote 
about the personal value of course topics in an introductory biology course for biomedical majors, and 
examined persistence in terms of subsequent course-taking and whether students were biomedical majors. 
The original study found that the UVI improved course grades for all students, on average, as well as for 
underrepresented students, by promoting engagement with intervention writing assignments. In the 
current follow-up, we found that the UVI indirectly increased persistence through the original effects on 
course grades, for all students on average and for underrepresented students in particular: better course 
grades were associated with higher levels of persistence. In addition, we found that among students who 
received the UVI, those who were more confident that they could succeed in the course were more likely 
to persist in a biomedical education 2 years later, an effect explained by increased focus on personal 
relevance in students’ biology writing assignments. This study has important implications for college 
science educators who seek to retain their students and promote diversity in the biomedical sciences. 
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Across a wide variety of occupations, there is an increasing 
demand for professionals with strong skills in the areas of science, 
technology, engineering, and mathematics (STEM; National Sci- 
ence Board, 2014). However, almost half (48%) of bachelor’s 
degree students who initially declare a major in a STEM field 
leave the STEM disciplines before completing college, thereby 
limiting their opportunities to develop these STEM skills (Chen, 
2013). Furthermore, the STEM workforce lacks diversity. Students 
from disadvantaged groups are less likely to obtain postsecondary 
STEM degrees than their majority peers. Underrepresented racial/ 
ethnic minorities (URMs) account for 26% of the population in the 
United States, yet they receive only 11% of college STEM degrees 
(National Science Board, 2014). First-generation (FG) college 
students (i.e., students for whom neither parent has a 4-year 
college degree) are also less likely to receive STEM degrees, 
dropping out of college at a much higher rate (28-35%) than their 
continuing-generation (CG) peers (17%; Radford, Berkner, Whee- 
less, & Shepherd, 2010). Given the importance of increasing the 
number of professionals in the STEM workforce and improving 
the representation of disadvantaged groups in these disciplines, it 
is crucial to test strategies to promote persistence in STEM fields. 
Psychologists can address these problems by designing interven- 
tions based in theories from social and developmental psychology 
(Harackiewicz & Priniski, 2018). 

In higher education, students’ decisions regarding whether to 
leave (or persist in) STEM often occur soon after taking a single 
introductory course, because this early experience shapes expec- 
tations for future STEM courses. College students who remain in 
a STEM field after an introductory course are more likely to later 
obtain a degree in that field (Chen, 2013). Therefore, by promoting 
students’ motivation and performance in an introductory course, an 
intervention may encourage students to persist beyond the course 
and set them on a positive trajectory toward obtaining a STEM 
degree. We investigate this possibility in the present study. 

One way to improve motivation and performance in college 
courses is by emphasizing the utility value (i.e., usefulness) of 
course material. This approach is based in Eccles’ expectancy- 
value theory of academic choices (Eccles & Wigfield, 2002), 
which posits that students’ academic decisions (e.g., which courses 
to take, which major to pursue) are determined by their expectan- 
cies for success and subjective task values (i.e., the perceived value 
or relevance of a task, including its utility value). To the degree 
that students perceive value in a field and believe that they can 
succeed, they are more likely to choose to pursue an education in 
that field. Furthermore, students’ expectancies for success and 
subjective task values are theorized to positively interact to predict 
their academic choices: students are most likely to pursue an 
education in a particular domain if they both value the domain and 
believe they can succeed in it. Longitudinal studies have found that 
expectancies for success and subjective task values are correlated 
with interest and engagement in academic courses, as well as the 
courses, majors, and careers that students choose to pursue (Har- 


ackiewicz, Durik, Barron, Linnenbrink-Garcia, & Tauer, 2008; 
Jacobs, Lanza, Osgood, Eccles, & Wigfield, 2002; Wang, 2012), 
and that effects are strongest among students with both high 
expectancies for success and high perceptions of task value (Guo 
et al., 2016; Nagengast et al., 2011). Expectancy-value theory 
predicts that an intervention that increases subjective task value in 
a particular domain should promote pursuit of that domain, par- 
ticularly among students who are confident that they can succeed. 
These confident students already believe they can pursue the 
domain, but an intervention that increases subjective task value 
may help them to see why pursuing that domain is worthwhile. 

Experimental studies show that students who receive a utility- 
value intervention (UVIJ), in which they complete writing assign- 
ments describing how academic material is relevant or useful, 
develop more interest and perform better (Canning & Harackie- 
wicz, 2015; Hulleman, Godes, Hendricks, & Harackiewicz, 2010). 
Recent field studies have demonstrated that this intervention can 
improve performance in high school and college STEM courses, 
particularly among students who have a history of poor academic 
performance or doubt their academic ability, indicating a negative 
interaction between the UVI and expectancies for success (Har- 
ackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016; Hulleman 
& Harackiewicz, 2009; Hulleman, Kosovich, Barron, & Daniel, 
2017). These findings are counterintuitive. In contrast to the pos- 
itive expectancy-value interaction theorized with respect to pro- 
spective academic decisions (e.g., choosing courses and majors), 
the negative expectancy-value interaction observed on perfor- 
mance (a proximal academic outcome) indicates a compensatory 
effect whereby reflecting on the value of course topics helps the 
students who are most likely to struggle in the course. Prior 
research suggests that generating examples of utility may help 
lower-performing students to become more interested in the ma- 
terial, which can promote learning (Hulleman et al., 2010; Hulle- 
man & Harackiewicz, 2009). This compensatory process may be 
driven by increased engagement with the course material, which 
leads to improved understanding and better performance. 

Indeed, research has provided support for an engagement-driven 
effect of the UVI on academic performance. Harackiewicz et al. 
(2016) conducted a large-scale randomized controlled trial testing 
the effectiveness of a UVI for closing achievement gaps in an 
introductory college biology course. They examined the intersec- 
tion of race and social class (defined in terms of FG status) and 
tested whether the UVI was particularly effective for URM (i.e., 
African American, Latino/a, and Native American) students, FG 
college students, and/or “FG-URM” students (who are both FG 
and URM; Cole, 2009). These groups face distinct sets of chal- 
lenges in college and tend to underperform compared with their 
CG-majority peers (Fiske & Markus, 2012; Lee, 2002). Therefore, 
students who are both FG and URM may be at a distinct disad- 
vantage (Jack, 2014). Harackiewicz et al. (2016) found that the 
UVlI improved course grades, averaged across all students, and this 
effect was especially strong for students with a history of poor 
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academic performance. Moreover, the UVI was particularly effec- 
tive for FG-URM students, improving their course grades by 0.51 
grade points and closing the achievement gap relative to their 
CG-majority peers by 61%. Students’ engagement in the writing 
assignments (as indicated by essay length) mediated these effects, 
providing evidence for one process by which the UVI improves the 
course performance of students who are most likely to struggle: by 
fostering engagement with the material. 

Although the UVI has been found to improve performance in 
introductory STEM courses, less is known about whether this 
intervention affects long-term decisions to pursue a STEM educa- 
tion. Could the UVI also encourage college students to take addi- 
tional STEM courses and even pursue a major in a STEM field? 
Canning and colleagues (2018) recently tested the effects of a UVI 
on college students’ persistence in STEM shortly after an intro- 
ductory course. They found that students who received the inter- 
vention in an introductory biology course obtained higher grades 
in the course, were less likely to abandon their intended STEM 
major by the end of the semester, and were more likely to take 
another biology course the following semester. These findings 
provide the first evidence that a UVI may be able to increase 
short-term persistence in STEM, in addition to course perfor- 
mance, and suggest that one route by which the UVI could pro- 
mote STEM degree attainment is by increasing the likelihood that 
students will enroll in another STEM course (see also Rosenzweig 
et al., 2018). However, the study is limited in that persistence was 
measured only one semester after the intervention, leaving ques- 
tions about longer-term impacts unanswered. Therefore, in the 
present research, we examined whether a UVI can influence aca- 
demic decisions over a longer time course. 

On the one hand, it may seem unlikely that a series of brief 
writing assignments focused on the relevance and usefulness of 
specific topics within a single course could have a long-term 
impact on students’ STEM persistence. However, there are at least 
two reasons to hypothesize that the benefits of the UVI would be 
detectable a year or more later. First, we know that the UVI can 
improve students’ grades in gateway introductory courses, and 
these improved grades may have downstream consequences. 
Higher grades in introductory courses may be a prerequisite for 
continuing on to higher-level courses, and may also influence 
students’ beliefs. A higher grade in a difficult introductory course 
can signal that students are on the right track and competent 
enough to continue on in the major, whereas a lower grade may 
lead students to conclude that they cannot make it, and consider 
changing their major to another field. Thus, the UVI may influence 
STEM persistence indirectly, by improving grades in introductory 
courses. Because the effects of the UVI on course grades are well 
established, this seems the most likely process through which 
long-term effects of the UVI may be detected. However, it is also 
possible that the UVI could have direct effects on long-term 
persistence. Expectancy-value theory predicts that students will be 
most likely to pursue a STEM field if they value the field and 
expect to be successful (Eccles & Wigfield, 2002). To the extent 
that the UVI helps students to focus on the value of STEM topics, 
it should increase STEM persistence, with the strongest effects for 
those who are confident that they can succeed. In summary, the 
UVI may influence long-term persistence directly, through 
expectancy-value processes, or indirectly, through higher grades in 
introductory courses. 


When examining the potential long-term effects of an interven- 
tion, it is important to consider the motivational processes by 
which the intervention may operate in the long term and whether 
they would be different from the processes that operate in the short 
term (Harackiewicz & Priniski, 2018). Whereas engagement with 
course material may promote course performance, we would not 
expect such a task-specific process to generalize beyond that 
particular course. In contrast, a more general sense of the personal 
relevance of science may encourage students to continue their 
education within the domain, especially among students who are 
already confident that they could do so successfully (Priniski, 
Hecht, & Harackiewicz, 2018). In other words, UVIs may initiate 
two motivational processes: task engagement and personal rele- 
vance, with different implications for different outcomes (perfor- 
mance in the short term vs. academic decisions in the longer term) 
and different populations (e.g., students who struggle vs. confident 
students). Task engagement may be especially important for stu- 
dents who struggle in the class, because it helps them learn more 
effectively, contributing to positive effects on performance, whereas 
personal relevance may more strongly influence the academic deci- 
sions of confident students. 

Linguistic analysis of students’ writing assignments may allow 
us to identify and test these two motivational processes (Beigman 
Klebanov, Burstein, Harackiewicz, Priniski, & Mulholland, 2017). 
Harackiewicz et al. (2016) documented the process by which the 
UVI promotes performance through engagement with course ma- 
terial by counting the number of words used in the assigned essays 
(i.e., essay length). They also determined the degree to which 
students were able to make specific connections between the 
material and their own lives with human coding (Canning et al., 
2018; Harackiewicz et al., 2016). However, more sophisticated 
techniques can provide insight into the ways in which students 
think about course topics more broadly. Indeed, research suggests 
that the words individuals use can capture thought processes that 
serve as powerful indicators of important outcomes, such as 
trauma recovery, mental and physical health, and academic suc- 
cess (Pennebaker, 2011; Pennebaker, Chung, Frazee, Lavergne, & 
Beaver, 2014). Linguistic Inquiry and Word Count (LIWC) soft- 
ware, for example, can be used to assess the degree to which 
students reflect on scientific concepts with regard to themselves or 
others by detecting the frequency of particular pronouns (Penne- 
baker, Booth, & Francis, 2007). This type of linguistic analysis 
may be particularly informative because it goes beyond the spe- 
cific topic connections that students make in an introductory 
science course and provides a more general indicator of the degree 
to which students think about science in personal terms. Such an 
indicator might allow researchers to assess the personal relevance 
process initiated by UVIs and could be key for understanding 
longer-term effects on STEM persistence. 


Current Study 


The goal of the present study is to examine two critical issues: 
the potential of a UVI to affect long-term pursuit of an education 
in STEM, and the mechanisms through which this may occur. A 
2-year follow up of the original Harackiewicz et al. (2016) study 
was conducted to address the following questions: (a) can the UVI 
promote persistence in a STEM field 2 years after implementation 
in an introductory course; and if so, (b) are long-term effects 
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stronger for particular groups of students (e.g., FG-URM students, 
confident students); and (c) are long-term effects driven by a 
different motivational process than the engagement process docu- 
mented for the short-term effects on performance? 

This research examined long-term effects of the UVI on persis- 
tence within the biological and medical sciences (hereafter bio- 
medical sciences). Retaining students in the biomedical sciences is 
particularly important because an influx of diverse professionals 
into these fields would help fuel innovation in medical research 
and treatment (Finkelstein & Hambrick, 1996). However, attrition 
is especially pernicious in the biomedical sciences, as students are 
23% less likely to remain in these fields throughout college com- 
pared with mathematics, engineering, and the physical sciences 
(Bettinger, 2002). The Harackiewicz et al. (2016) study was con- 
ducted in an introductory biology course taken by prospective 
majors and prehealth students. This study was extended by exam- 
ining whether the intervention produced positive effects on per- 
sistence in the biomedical track. Specifically, we investigated 
effects on persistence soon after the biology course (i.e., enroll- 
ment in the next course in the biology sequence) and 2 years later 
(i.e., majoring in a biomedical field 2 years postintervention). 


Method 


Participants 


The UVI was implemented in the first course of a two-course 
introductory biology sequence in a double-blind randomized con- 
trolled trial at a large Midwestern university. This sequence is one 
option for fulfilling introductory biology requirements at the uni- 
versity. It serves as a primary pathway to premedical preparation 
and postsecondary study in the biological sciences because it is 
designed specifically for biomedical sciences students, and the first 
course is a prerequisite for the second. A total of 2,378 students 
(8% URM, 21% FG) were enrolled in the course over the four 
semesters in which the intervention was conducted. All consenting 
FG and URM students were included in the study, along with a 
randomly selected subset of CG-majority students (82% White, 
18% Asian or Asian American). Of the 1,060 students eligible for 
the study, 1,039 completed the course and gave consent for access 
to their academic records (15 students dropped the course, and 6 
students did not consent). The sample for the present study is 
comprised of these 1,039 students (60% women), of whom 422 
were CG-majority, 427 were FG-majority, 126 were CG-URM, 
and 64 were FG-URM.' 579 of these students were assigned to the 
UVI condition and 460 were assigned to the control condition. Of 
these 1,039 students, 918 (88%) took the course in their second 
year of college. 


Procedure 


This follow-up study tracked students over the 2 years after the 
semester in which they received the UVI. Students had been 
randomly assigned within lecture sections to the UVI condition or 
the control condition, stratified by gender, FG, and URM status. 
Instructional staff in the course were all blind to experimental 
condition. In both conditions, students were assigned to complete 
three writing assignments throughout the course of the semester, 


presented as graded course assignments. Students in the control 
and UVI conditions were both asked to: 


Select a concept or issue that was covered in lecture and formulate a 
question. Select the relevant information from class notes and the 
textbook, and write a 1-2 page essay. 


In the control condition, students were asked to answer their 
question by summarizing material from the course: 


Select the relevant information from class notes and the textbook, and 
write a one to two-page response to your question. You should attempt 
to organize the material in a meaningful way, rather than simply 
listing the main facts or research findings. Remember to summarize 
the material in your own words. 


Writing assignments in the UVI condition varied somewhat 
throughout the study, but in each assignment, students were asked 
to address their question using material from the course and write 
about the relevance of the topic. In an essay variation, students 
were asked to connect the course material to their own life: 


Write an essay addressing this question and discuss the relevance of 
the concept or issue to your own life. Be sure to include some concrete 
information that was covered in this unit, explaining why this specific 
information is relevant to your life or useful for you. Be sure to 
explain how the information applies to you personally and give 
examples. 


In a letter variation of the UVI, students were asked to connect 
the material to other individuals: 


Write a letter to a family member or close friend, addressing this 
question, and discuss the relevance of this specific concept or issue to 
this other person. Be sure to include some concrete information that 
was covered in this unit, explaining why the information is relevant to 
this person’s life or useful for this person. Be sure to explain how the 
information applies to this person and give examples. 


Biology graduate students graded the assignment for scientific 
accuracy and adherence to instructions. The assignments were 
cumulatively worth 1.8% of the grade in the course (0.6% per 
assignment). This study was approved by the Institutional Review 
Board. 


Measures 


Measures were collected at five time points: (a) during the first 
week of the introductory biology course, before administering the 
intervention (FG and URM status, prior grade point average 
[GPA], confidence about performance in the course, baseline ma- 
jor), (b) during the course (linguistic measures computed from the 
three writing assignments: engagement and personal focus), (c) at 
the end of the course (course grade), (d) the next semester (enroll- 
ment in the second introductory biology course), and (e) 2 years 
postintervention (persistence in a biomedical major; see Figure 1). 


! This number differs from the 1,040 students included in the Harack- 
iewicz et al. (2016) study. Institutional data collected for the follow-up 
study indicated that one student had received a retroactive withdrawal from 
the course after the data were originally released to researchers. Excluding 
this student does not change the results of the original study, and including 
this student does not change the results of the present study. 
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During the Course 
Utility-value intervention 
(3 assignments) 
Engagement 
Personal focus 


Beginning of Semester 
Prior GPA 
Confidence about performance 


End of Course 


Course grade 


Two Years Post- 


Intervention 
Biomedical major (yes/no) 


Next Semester 
Enrollment in second 
course (yes/no) 


Figure 1. 


Zero-order correlations between measures are presented in Ta- 
ble 1. 

Baseline measures. In a questionnaire administered at the 
beginning of the course, students reported the highest level of 
education achieved by their parents. Students with at least one 
parent who had obtained a bachelor’s degree or higher were 
classified as CG (continuing-generation), and all other students 
were classified as FG (first-generation). Students also reported 
their race/ethnicity; those who self-identified as African American, 
Latino/a or Native American were classified as URM (underrep- 
resented minority), and all other students were classified as ma- 
jority. Three questionnaire items (“I am confident that I will do 
well in Introductory Biology;” “I expect to get a good grade in this 
course;” “I am confident that I can obtain a final grade of B or 
better in this course”) were combined to create the measure of 
confidence about performance (a = .82). Students were also asked 
to report their intended or current major. Students were coded as 
being in a biomedical major at baseline if (a) their intended major 
was in a biomedical field, or (b) they indicated that they were on 
a prehealth educational track (i.e., premedical, predental, prephar- 
macy, preveterinary, and preoptometry). According to these crite- 
ria, 972 students (94% of the sample) were categorized as being in 
a biomedical major at baseline. Coding of academic majors for all 
students in the sample is provided in supplemental Table 5. 

Prior GPA and course grades. Students’ GPAs from previ- 
ous semesters were obtained from institutional records.” Final 
grades in the course were provided by course instructors at the end 
of the semester. Instructors calculated grades from students’ per- 
formance in lecture, discussion, and laboratory sections including 
multiple-choice and short-answer exams and quizzes (60%), lab- 
oratory activities (35%), and discussion activities (5%). The pres- 
ent sample comprises eight lecture sections across four semesters, 
and course coordinators worked to ensure standardization of con- 
tent and grading procedures across sections and semesters. Grades 
were assigned on a 4.0 scale: A = 90-100% (4.0 grade points), 
AB = 88-89.99% (3.5 grade points), B = 80—87.99% (3.0 grade 
points), BC = 78-79.99% (2.5 grade points), C = 70-77.99% 
(2.0 grade points), D = 60-69.99% (1.0 grade points) and 
F = <60% (0 grade points). Course grades were not curved. 

Engagement and personal relevance. We used two linguis- 
tic measures to assess the processes initiated by the UVI: engage- 
ment and personal relevance. We used Harackiewicz et al.’s 
(2016) measure of engagement with the writing assignments by 
averaging the number of words used across the three writing 
assignments. In terms of personal relevance, prior work has used 
either a binary complier index, which assesses whether or not 
students made a utility value connection in each assignment, as a 
manipulation check (Hulleman & Cordray, 2009), or a measure of 
articulated utility value, which assesses the quality of the specific 
utility-value connections made in the assignments (Canning et al., 
2018; Harackiewicz et al., 2016); these two measures were highly 


Timeline of measures taken throughout the study. 


correlated (r = .81) in this sample. However, because students in 
UVI conditions were explicitly asked to write about utility value, 
both the binary complier and articulated utility value measures are 
best conceptualized as manipulation checks. 

For the current study, we examined a linguistic measure of 
personal relevance, because such measures capture differences in 
the way students write, rather than the content of their writing. 
Specifically, we developed a measure of “personal focus,” or the 
degree to which students were thinking and writing about biology 
in terms of its relation to their own or others’ lives. We measured 
personal focus by assessing the extent to which students wrote 
about science in personal terms by examining the number of 
first-person singular and second-person pronouns (e.g., J, my, you, 
your) as a proportion of total words in students’ essays using 
LIWC (Pennebaker et al., 2007). As a proportional measure, 
personal focus indicates the degree to which students’ biology 
writing assignments were saturated with personal content. 
Whereas the binary complier index was designed to maximally 
distinguish between UVI and control writing assignments, and 
articulated utility value was designed to measure the quality of 
connections in the sentences in which students discussed utility 
value, the personal focus measure goes beyond specific utility- 
value connections to provide a more general indicator of the 
degree to which students wrote about science in personal terms, 
across the entire essay.* The measures of engagement and personal 
focus were independent (r = .02). 

Enrollment in the second biology course. Students intending 
to major in the biomedical sciences typically take two or three 
introductory-level biology courses, and students who take the 
course in which the UVI was implemented typically take the 
second course in that sequence (often the next semester). Thus, 
enrollment in the second course is a good indicator of progression 
toward timely completion of a biomedical degree. We obtained 
enrollment data from institutional records. A total of 842 students 
(81% of the sample) enrolled in the second biology course; of 
these students, 91% did so the semester immediately following the 
first course. 

Persistence in a biomedical major. Students’ majors were 
obtained from institutional records four semesters postinterven- 
tion. We measured persistence in a biomedical major at this point 
in time because most students would be in their senior year of 


? There were 61 freshmen or transfer students in the sample, and prior 
GPA was, therefore, imputed for these individuals (see online supplemen- 
tal materials for handling of missing data). 

>We conducted ancillary analyses to test two alternative measures 
designed to account for differences in pronoun usage between essays and 
letters. We found the same pattern of effects as reported in the results 
section below, suggesting that results were not attributable to differential 
pronoun usage in letters and essays. See the online supplemental materials 
for a detailed report. 
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Table | 
Zero-Order Correlations 


Variable 1 2 3 4 5 6 7 
1. Confidence about performance —_ 
2. Prior GPA Pes ae —: 
3. Course grade 23°" 538°" — 
4. Personal focus .00 01 =,01 —_ 
5. Engagement 06" 2 2 02 — 
6. Enrollment in second biology course 10°" 3°" 2 O01 OT" — 
7. Biomedical major .09** Ap 1°" —.05 .08* 58" —_— 
Note. GPA = grade point average. 


“p< .05. 


% 


“p< .0l. ““p < .001. 


college or would have graduated. Indeed, 85% of the sample were 
seniors or had graduated by the time of this follow-up, and 10% 
were juniors. A small proportion of the sample (3%) was identified 
as having left the institution (e.g., transferred, dropped out),* and 
the remaining 1% were sophomores. 

Majors were coded as biomedical if the student’s major was in 
a biomedical field or if the student was still on a prehealth 
educational track (e.g., a premedical student with a psychology 
major).° A total of 793 students (76%) were categorized as being 
in a biomedical major 2 years postintervention: 743 (88%) of the 
842 students who took the second introductory biology course and 
50 (25%) of the 197 students who did not take the second course 
(see Table 2). 


Results 


Analysis Plan 


Analyses were designed to test the effects of the UVI on 
persistence in a biomedical field 2 years postintervention, and the 
motivational processes through which the intervention has its 
effects. We examined whether (and for whom) the UVI had direct 
effects on measures of persistence (enrollment in the second biol- 
ogy course [or not] and biomedical major [or not], 2 years postin- 
tervention). We also tested whether the intervention indirectly 
affected persistence, via the more proximal effects on course 
performance reported by Harackiewicz et al. (2016). Finally, we 
examined the processes by which the UVI might promote long- 
term persistence. Specifically, we tested whether linguistic indica- 
tors of engagement and personal focus would mediate any long- 
term effects of the intervention. 

Path models using maximum likelihood estimation with a logit 
function were used to examine effects on the binary persistence 
outcomes: enrollment in the second biology course (enrollment = 


Table 2 
Frequencies of Enrollment in the Second Biology Course X 
Biomedical Major 


Enrollment in second biology course 


Persistence in 


biomedical major No Yes 
No 147 (14%) 99 (10%) 
Yes 50 (5%) 743 (72%) 


1, no enrollment = 0) and biomedical major (biomedical major = 
1, nonbiomedical major = 0) 2 years postintervention.° Coeffi- 
cients for effects on these binary outcomes represent log-odds. 
Predicted probabilities are displayed graphically to aid interpreta- 
tion. Additional details about our analytic approach and model fit 
can be found in the online supplemental materials. 


Direct Intervention Effects on Persistence 


The effects of the UVI were tested on two measures of persis- 
tence (enrollment in the second biology course and biomedical 
major) using the same set of independent variables that Harackie- 
wicz et al. (2016) used to predict course grade: UVI (UVI = 1, 
control = —1), FG status (FG = 1, CG = —1), URM status 
(URM = 1, majority = —1), all two-way interactions and the 
three-way interaction between these variables, prior GPA (stan- 
dardized), confidence about performance (standardized), and the 
two two-way interactions between the UVI and each of these 
variables. Additionally, baseline major (biomedical = 1, nonbio- 


medical = —1) was included as a covariate to control for baseline 
differences in majoring intentions; this was the basic model for all 
analyses. 


The main effect of the UVI was not significant for either 
enrollment in the second course or biomedical major 2 years 
postintervention, in contrast to the significant main effect on 
course grade originally observed in this study (Harackiewicz et al., 
2016), and in contrast to the significant main effect on enrollment 
in a second course in a different sample (Canning et al., 2018). 
However, there was a significant UVI X Confidence interaction on 
both enrollment in the second biology course (z = 2.08, b = 0.18, 
Pp = .04) and biomedical major 2 years postintervention (z = 2.62, 
b = 0.21, p = .01; see Figure 2). This pattern of effects suggests 
that the UVI moderated the degree to which confidence influenced 
students’ persistence in the biomedical track. To probe this possi- 


“We identified students as having left the institution if they were not 
enrolled for at least two semesters postintervention. The proportion of 
students in this category did not significantly differ by condition. 

° There were 1,024 students’ majors coded according to these criteria. 
There were 15 students (1% of the sample) for whom we had incomplete 
major data (e.g., because they had not yet declared a major). These students 
were coded as not having a biomedical major. 

© Given our prioritization of testing the structure of variable relationships 
over an ideal measurement model, we opted to test path models rather than 
structural equation models, treating confidence about performance as ob- 
served using factor scores. 
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Figure 2. Utility-value intervention (UVI) X Confidence about performance effect on the probability of 
enrolling in the second biology course (panel A) and persisting in a biomedical major (panel B). There was no 
effect of confidence on enrollment (p = .74) or major (p = .24) for students in the control condition, but for 
students who received the UVI, confidence positively predicted both enrollment (p < .01) and major (p < .01). 


bility, we tested simple effects of confidence for students in the 
control and UVI conditions. In the control condition, confidence 
about performance did not significantly affect enrollment (z = —.34, 
b = —.04, p = .74) or major (z = —1.17, b = —.14, p = .24), but 
in the UVI condition, there was a significant positive effect of con- 
fidence on enrollment (z = 2.75, b = .31, p < .01) and major (z = 
2.67, b = .28, p < .01). These findings suggest that confidence played 
an important role in decisions about whether to persist in the biomed- 
ical track for students who were asked to reflect on the value of course 
material, but not for students who were not prompted to engage in 
such reflection. We further probed this interaction by testing simple 
effects of the UVI at high and low levels of confidence (+1 SD). The 
effects of the UVI were more positive for students with higher levels 
of confidence (z = 1.74, b = .25, p = .08 for enrollment; z = 1.43, 
b = 19, p = .15 for major) than for students with lower levels of 
confidence (z .80, b .11, p = 42 for enrollment; z = —1.73, 
b = —.23, p = .08 for major), but none of these simple effects were 
significant. 

Students with higher prior GPAs and students who entered the 
course intending to pursue a biomedical major were more likely to 
enroll in the second course (ps < .001). URM students were less 
likely to enroll in the second biology course than majority students 
(p = .01). This pattern of effects also emerged on biomedical 
major, except that the URM effect was not statistically significant 
(p = .17). Neither FG nor URM status significantly interacted with 
the UVI to predict enrollment or major, suggesting that the UVI 
did not increase persistence differentially for FG or URM students 
(ps > .25; see Table 3 for full model output). 

We also examined whether these two significant intervention 
effects were sequential, such that promoting enrollment in the next 
course for confident students led to long-term persistence in a 
biomedical major for confident students. We tested a second path 
model adding enrollment in the second course as a predictor of 
biomedical major. The pattern of effects was identical to those 
found in the previous analysis (see Table 4), but we also found a 
strong effect of enrollment in the second course on biomedical 
major (z = 13.96, b = 2.88, p < .001). Students who enrolled in 


the second course were 58.22% more likely to major in biomedical 
fields than students who did not enroll in the second course. 
Indeed, we found that the UVI X Confidence effect on biomedical 
major was partially mediated by enrollment in the second course 
(z = 2.06, p = .04), such that within the UVI condition, students 
with higher confidence were more likely to enroll in the second 


Table 3 
Effects From the Model Testing Direct Intervention Effects on 
Persistence in the Biomedical Track 


Variable b Zz Dp 
Enrollment in second course 
UVI 07 .64 52 
FG = 01 —.09 93 
URM =27 = 2:55 01 
UVI X FG = 14 09 32 
UVI X URM —.02 =21 83 
FG X URM .02 al'5 88 
UVI X FG X URM =103 = 28 78 
Prior GPA 30 3.59 .00 
UVI X Prior GPA .00 .00 1.00 
Confidence about performance 13 157. 12 
UVI X Confidence about performance 18 2.08 04 
Baseline major 1.19 8.40 00 
Biomedical major 
UVI —.02 = 18 86 
FG 12 1.16 25 
URM —.14 ca eos 17 
UVI X FG —.06 352, .60 
UVI X URM .00 =,03 98 
FG X URM 2 1.19 23 
UVI X FG k URM =,12 =1,12 26 
Prior GPA Al 5.13 .00 
UVI X Prior GPA 03 39 69 
Confidence about performance 07 83 Al 
UVI X Confidence about performance 21 2.62 01 
Baseline major 1.21 8.13 00 


Note. FG = first-generation college student status; URM = underrepre- 
sented racial/ethnic minority status. 
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Table 4 

Effects From the Model Testing Enrollment in the Second 
Course as a Mediator of the Intervention Effect on Persistence 
in a Biomedical Major 


Variable b Zz Dp 
Enrollment in second course 
(Effects on enrollment are identical to 
those in Table 3) 
Biomedical major 
UVI —.06 —.49 62 
FG 7 1.36 AT 
URM .00 —.02 98 
UVI X FG .00 =102 99 
UVI X URM 01 a 92 
FG X URM 16 1,32 19 
UVI X FG X URM —.14 — 1.09 28 
Prior GPA ei 3.89 .00 
UVI X Prior GPA 0S 05 58 
Confidence about performance O01 .09 93 
UVI X Confidence about performance 19 1.98 048 
Enrollment in second course 2.88 13.96 .00 
Baseline major 80 4.46 .00 
Note. FG = first-generation college student status; URM = underrepre- 


sented racial/ethnic minority status. 


course, and enrollment in turn in turn increased the probability of 
persisting in a biomedical major 2 years postintervention. 

The finding that the UVI moderated the effects of confidence on 
long-term persistence is consistent with the theorized positive 
expectancy-value interaction, which posits that students must both 
perceive value and expect to succeed to be motivated to pursue a 
particular academic path. In the UVI condition, students were 
encouraged to reflect on the value of course content three times 
across the semester, and it was in this condition that more confi- 
dent students were significantly more likely to persist, which is 
consistent with a synergistic Expectancy X Value effect. As re- 


A 


Confidence 
about 
Performance 


Enrollment in 
Second 
Course 


Personal 
Focus 


Utility-Value 
Intervention 


ported in Harackiewicz et al. (2016), there were no significant 
effects of FG or URM status on baseline confidence, suggesting 
that this UVI < Confidence effect pertained to all students, re- 
gardless of FG and URM status. We did not find any direct effects 
of the UVI for FG-URM students. 


Testing Linguistic Mediators of Intervention Effects 


We next examined linguistic indicators of (a) personal focus and 
(b) engagement as possible mediators of the UVI < Confidence 
effect on persistence in the biomedical track. We reasoned that the 
UVI should lead students to write about science content in a more 
personal way (i.e., with a more personal focus), which in turn 
might predict persistence in the biomedical track for more confi- 
dent students. We also tested our measure of engagement (i.e., 
essay length) as a potential mediator, to determine whether the 
same process that explained short-term intervention effects on 
course performance in the Harackiewicz et al. (2016) study might 
also play a role in the long-term intervention effects on persistence. 

Separate moderated mediation models were estimated for each 
of these possible mediators, testing the basic model on each 
mediator, and then testing the effect of the mediator and the 
Mediator < Moderator (Confidence) interaction on enrollment. 
The personal focus model included a main effect of the UVI on 
personal focus (z = 28.09, b = 0.77, p < .001), as well as a 
significant Personal Focus < Confidence interaction (z = 2.43, 
b = 0.33, p = .02) indicating that personal focus had a more 
positive effect on enrollment in the second course for students who 
were more confident (see Figure 3 and Table 5). Indeed, personal 
focus partially mediated the UVI < Confidence effect on enroll- 
ment (z = 2.42, p = .02) and the UVI X Confidence effect on 
biomedical major, through enrollment (z = 2.39, p = .02). All 
other effects were consistent with those reported from the basic 
model, with the exception that the UVI < Confidence effects on 
enrollment and biomedical major became weaker, as would be 
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Figure 3. Panel A displays the conceptual moderated mediation model of utility-value intervention (UVI) 
effects on persistence in the biomedical track via personal focus, using an oval to depict the linguistic process 
variable (i.e., personal focus). Panel B displays the effect of confidence on enrollment in the second biology 
course at high (+.75 SD) and low (—.75 SD) levels of personal focus. 
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Table 5 

Effects From the Model Including Personal Focus as a 
Mediator of Intervention Effects on Persistence in the 
Biomedical Track 


Variable b Zz Dp 
Personal focus 
UVI 4 28.09 .00 
FG OL 21 83 
URM .03 1.32 19 
UVI X FG 04 125 21 
UVI xX URM 02 70 49 
FG X URM 02 .66 S1 
UVI X FG X URM 04 1.47 14 
Prior GPA .00 = 21 84 
UVI X Prior GPA OL 64 so: 
Confidence about performance .00 .10 92 
UVI X Confidence about performance O01 38 70 
Baseline major O1 69 49 
Enrollment in second course 

UVI 06 40 69 
FG OL .05 96 
URM 28 =2:65 01 
UVI X FG =109 —.82 Al 
UVI X URM —.02 =21 84 
FG X URM 02 16 87 
UVI X FG X URM =,03 = 28 78 
Prior GPA 30 3.64 .00 
UVI X Prior GPA OL 12 91 
Confidence about performance 16 1.86 06 
UVI X Confidence about performance —.08 —.62 54 
Personal focus 03 18 86 
Personal Focus X Confidence about 

performance 33 2.43 02 
Baseline major 1.20 8.47 00 


Biomedical major 
(Effects on major are identical to those 
in Table 4) 


Note. FG = first-generation college student status; URM = underrepre- 
sented racial/ethnic minority status. 


expected with a significant moderated mediation model. Engage- 
ment did not mediate the direct effects on persistence (ps > .13).” 

The analyses thus far suggest that personal focus is the more 
important mediator for understanding long-term effects of the 
UVI, but is this process independent from the process by which the 
UVI promotes short-term performance through engagement, or 
might personal focus also play a role in intervention effects on 
performance? To investigate this question, we tested the same 
moderated mediation model used in the original Harackiewicz et 
al. (2016) study but also included personal focus as a potential 
mediator. Consistent with the original findings, we found that 
engagement mediated the main effect of the UVI (z = 2.38, p = 
.02) and the UVI X FG X URM effect (z = 2.56, p = .01) on 
course grade, but personal focus was not a significant mediator of 
either of these effects (ps > .13).8 

Taken together, these findings suggest that the UVI can affect 
two distinct outcomes—short-term performance and long-term 
academic decisions—by influencing two different motivational 
processes. The UVI X Confidence interaction on long-term per- 
sistence was mediated by personal focus, suggesting that the UVI 
prompted students to write about science in a more personal way, 
which in turn led more confident students to remain in the bio- 


medical track. Conversely, the finding that the effect of the UVI on 
course grade (overall, and particularly for FG-URMs) was medi- 
ated by engagement suggests that to the extent that these students 
engaged more with specific material from the course, the UVI 
helped them to improve their performance in that course. 


The Role of Course Performance in Long-Term 
Persistence 


Whereas the UVI did not differentially affect persistence in the 
biomedical track for FG-URM students, it is possible that the UVI 
effect on course grade for these students might promote perfor- 
mance indirectly. Therefore, we next assessed the degree to which 
improved grades predicted long-term persistence, controlling for 
the direct effects of the intervention on persistence found in our 
prior analyses (see Figure 4 for a conceptual diagram). We started 
with our previous model and added biology course grade (as 
predicted by the basic model and engagement) as a predictor of 
enrollment in the second course and biomedical major. Grade in 
the first course had a strong effect on both measures of persistence: 
a change of one standard deviation in grade corresponded, on 
average, to a 10.24% increase in the predicted probability of 
enrolling in the second course (z = 6.71, b = 0.73, p < .001) and 
an 8.44% increase in the predicted probability of enrolling in a 
biomedical major (z = 5.08, b = 0.61, p < .001; see Table 6). 

Indirect effects. Given the strong effects of course grade on 
persistence, it follows that the UVI effect on course grade for 
FG-URM students might have had long-term implications for 
persistence. If so, we would expect a significant indirect effect of 
the UVI on persistence through engagement and performance for 
these students in this model. Indeed, we found a significant indi- 
rect effect of the UVI on enrollment in the second course via 
engagement and course grade (z = 2.25, p = .02) for all students 
on average, as well as a significant indirect effect of the three-way 
UVI X FG X URM interaction on enrollment via engagement and 
course grade (z = 2.40, p = .02). Conditional indirect effects 
revealed a significant effect for FG-URM and CG-majority stu- 
dents, but not FG-majority and CG-URM students (see Table 7). 
Similarly, we found a significant indirect effect of the UVI (z = 
2.22, p = .03), as well as the UVI X FG X URM interaction (z = 
2.36, p = .02) on biomedical major via engagement, course grade, 
and enrollment in the second course. We found significant condi- 
tional indirect effects for FG-URM and CG-majority students, but 
not for FG-majority and CG-URM students (see Table 7). These 
effects suggest that to the extent that the UVI improved engage- 
ment and course grades for FG-URM and CG-majority students, it 
was associated with long-term persistence. 

To illustrate the implications of these indirect effects, Figure 5 
shows the UVI effects as a function of specific course grade 


7 We also tested a moderated mediation model including personal 
focus and engagement simultaneously as mediators. These tests also 
revealed that personal focus mediated the UVI X Confidence effects on 
enrollment (z = 2.20, p = .03) and major (z = 2.17, p = .03), but 
engagement did not (ps > .15). 

® We also tested engagement and personal focus in separate moderated 
mediation models and found that engagement significantly mediated the 
main effect of the UVI (z = 2.39, p = .02) and the UVI X FG X URM 
effect (z = 2.57, p = .01) on course grade, but personal focus did not 
significantly mediate these effects (ps > .09). 
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Figure 4. Conceptual model of utility-value intervention (UVD effects on persistence in the biomedical track 
via personal focus, engagement, and course grade, using ovals to depict linguistic process variables (i.e., personal 
focus, engagement). The dashed arrows represent indirect effects of the UVI on enrollment in the second course 


and biomedical major via engagement and course grade. 


categories (Panels A and B) for the full sample and the FG-URM 
students, respectively, and Panels C and D show the rates of 
enrollment and biomedical majoring associated with each grade 
category, for the full sample and specifically for FG-URM stu- 
dents, respectively. Students in UVI conditions (and FG-URM 
students in particular) were more likely to obtain grades in the 
AB-B range and less likely to obtain grades in the BC-C range, 
suggesting that the UVI effect was most pronounced in this range. 
Higher grades were in turn associated with a higher probability of 
persisting in the biomedical track, and these differences were most 
pronounced between the BC-C and AB-B range (i.e., the grade 
category ranges in which the UVI had the most pronounced 
effects). 

Sensitivity analysis for indirect effects via course grade. 
Given that we did not find direct effects of the UVI on persistence 
in the biomedical track that mirrored the UVI effects on course 
grade (i.e., a main effect of UVI or a UVI X FG X URM 
interaction), we cannot conclude that grade was a mediator of UVI 
effects. The indirect effects of the UVI on persistence via course 
grade point to a potential pathway through which improved grades 
might promote persistence, but it is possible that an unmeasured 
confounding variable accounts for these indirect intervention ef- 
fects. Therefore, we conducted a sensitivity analysis to test the 
proportion of variance in the mediator and outcome that would 
need to be accounted for by an unmeasured confounding variable 
to reduce the indirect effect to zero (Imai, Keele, & Yamamoto, 
2010).? We found that an unmeasured confounding variable would 
have to account for at least 13.67% of the variance in both course 


grade and enrollment in the second course or 19.87% of the 
variance in both course grade and major to reduce the indirect 
effects of the UVI via course grade to zero. These results suggest 
that the indirect effects reported on persistence in the biomedical 
track via course grade are fairly robust to unmeasured confounding 
variables. 


Discussion 


We examined the role of a utility-value intervention in promot- 
ing persistence in the STEM pipeline at the postsecondary level, 
addressing the need to reduce attrition from biomedical fields. 
Harackiewicz et al. (2016) tested this intervention in a gateway 
biology course taken by students intending to major in the bio- 
medical sciences, and we followed these students for 2 years after 
they completed the course. Harackiewicz et al. (2016) had found 
that the UVI improved course grades, averaged across all students 


° There are presently several limitations to available methods for con- 
ducting sensitivity analyses: it is not currently possible to conduct sensi- 
tivity analyses for moderated mediation models, nor is it possible to test 
sequential mediation models. Therefore, we tested the following simple 
mediation models: (a) a model examining the main effect of the UVI on 
enrollment in the second course via course grade, and (b) a model exam- 
ining the main effect of the UVI on biomedical major via course grade. For 
each model, we included the basic model on course grade and the measure 
of persistence, and the effect of course grade on the measure of persistence. 
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Variable b Zz Dp 
Personal focus 
(Effects on personal focus are identical 
to those in Table 5) 
Engagement 
UVI AY 2.66 01 
FG 02 AT 64 
URM 04 sie all 
UVI X FG 05 1.09.27 
UVI X URM 06 1:39: .17 
FG X URM .00 05 .96 
UVI X FG X URM 12 2.91 — .00 
Prior GPA .20 6.15 .00 
UVI X Prior GPA OL AT 387 
Confidence about performance 04 £19 3.23 
UVI X Confidence about performance OL 33>, a7) 
Baseline major .00 07 = 95 
Course grade 
UVI 06 1.84  .07 
FG 03 —.96 34 
URM ks) —4.97 — .00 
UVI X FG .00 08 = .93 
UVI xX URM 05 158 11 
FG X URM .00 13.90 
UVI X FG X URM 07 2.01 —.045 
Prior GPA 03 20.49 = .00 
UVI X Prior GPA 06 —2.24 03 
Confidence about performance 14 5.71.00 
UVI X Confidence about performance 03 1.28 = .20 
Engagement 13 5.44 00 
Baseline major 03 1.06  .29 
Enrollment in second course 
UVI 02 =12:- .,91 
FG 03 23 82 
URM 16 —142 16 
UVI X FG 11 =99 .32 
UVI X URM 06 —.58 56 
FG X URM OL ll 92 
UVI X FG X URM 10 —.89 37 
Prior GPA 09 —.86  .39 
UVI X Prior GPA 06 69 49 
Confidence about performance 06 61 54 
UVI X Confidence about performance 07 —49 63 
Personal focus 08 53.60 
Personal Focus X Confidence about 
performance .29 1.98  .048 
Course grade .73 6.71 — .00 
Baseline major .24 8.38 00 
Biomedical major 
UVI 10 = 15.- 5 
FG 21 162 .11 
URM 10 19 43 
UVI X FG .00 —.02 98 
UVI X URM 02 —.18 86 
FG X URM 18 1.37.17 
UVI X FG X URM 18 ST «Ait 
Prior GPA 05 Al 68 
UVI X Prior GPA 10 97 33 
Confidence about performance 09 —.91 36 
UVI X Confidence about performance 18 1.81 .07 


Table 6 

Direct Effects From the Model Including Personal Focus, Variable b 7 P 

Engagement, and Course Grade as Potential Mechanisms of Course grade 61 5.08 00 

UVI Effects on Persistence in the Biomedical Track Enrollment in second course 2.72 12.84 .00 
Baseline major 85 4.65 .00 


Note. FG = first-generation college student status; URM = underrepre- 
sented racial/ethnic minority status. 


and particularly for FG-URM students, and that this intervention 
effect was mediated by engagement. The present study extended 
these findings by examining whether the UVI also played a role in 
promoting students’ persistence in the biomedical track 2 years 
beyond this initial course, as well as the psychological processes 
that may underlie these effects. 

We found both direct and indirect effects of the UVI on long- 
term persistence. As predicted, the UVI had indirect effects by 
improving course performance, which in turn had strong effects on 
persistence, and these indirect effects were strongest for FG-URM 
students. As shown in Figure 5, the intervention was most effective 
in changing grades between the BC-C and AB-B ranges, and such 
grade differences can be psychologically significant. A student 
who receives a C in a critical class may experience their grade as 
a signal that they cannot make it, whereas a student who gets a B 
may view their grade as a signal that they are on track. By 
improving performance in a critical gateway course, the UVI may 
have helped students meet requirements for their major or in- 
creased their belief that they could succeed in a biomedical field. 
This type of process may be especially important for groups who 
struggle in STEM courses (e.g., FG-URM students). The UVI 
partially closed the achievement gap in the introductory biology 
course for FG-URM students, and the significant conditional in- 
direct effects on course-taking and majoring decisions for this 
group suggest that UVIs may be a useful tool for promoting 
diversity in the biomedical fields. 

A single intervention in one class may not be sufficient to 
promote long-term persistence directly for struggling students, but 
it may get them “over the hump” of gateway courses through this 
process. However, if subsequent courses are not perceived as 
valuable, or if other supports are not available (e.g., mentoring, 
research experiences; Hernandez, Schultz, Estrada, Woodcock, & 
Chance, 2013; Russell, Hancock, & McCullough, 2007; Schultz et 
al., 2011), the positive effects of the UVI may be outweighed by 
other challenges and dissipate over time. With respect to pro- 
moting diversity in the biomedical sciences, the UVI should 
be viewed as a first step in this multistage process. It will be 
important to consider how the UVI in gateway courses might be 
combined with “booster” interventions in more advanced 
courses, or with other mentored research and advising programs 
for advanced students. 

We also found direct effects on persistence, as a function of 
confidence. Specifically, we found direct effects of the UVI, 
moderated by students’ confidence levels, on enrollment in a 
second introductory biology course as well as persistence in a 
biomedical major 2 years postintervention. Simple effects anal- 
yses revealed that confidence positively predicted persistence 
for students who received the UVI, over and above the effect of 
prior GPA, but not for students in the control condition. These 
findings suggest that in a context that encourages reflection on 
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Table 7 


Indirect Effects From the Model Including Personal Focus, Engagement, and Course Grade as Potential Mechanisms of UVI Effects 


on Persistence in the Biomedical Track 


Indirect effect Zz Dp 
UVI — Personal Focus X Confidence Enrollment in second course 1.98 048 
UVI — Personal Focus X Confidence — Enrollment in second course — Biomedical major 1.95 051 
UVI — Engagement — Course grade 2.39 02 
UVI — Engagement — Course grade (CG-majority) 2.37 02 
UVI — Engagement — Course grade (CG-URM) .03 98 
UVI — Engagement — Course grade (FG-majority) —.47 64 
UVI — Engagement — Course grade (FG-URM) 2.43 02 
UVI X FG X URM — Engagement — Course grade 2.57 01 
UVI — Engagement — Course grade — Enrollment in second course 2:25 02 
UVI — Engagement — Course grade — Enrollment in second course (CG-majority) 2.24 03 
UVI — Engagement — Course grade — Enrollment in second course (CG-URM) .03 98 
UVI — Engagement — Course grade — Enrollment in second course (FG-majority) — 47 64 
UVI — Engagement — Course grade — Enrollment in second course (FG-URM) 2.28 02 
UVI X FG X URM — Engagement — Course grade — Enrollment in second course 2.40 02 
UVI — Engagement — Course grade — Enrollment in second course — Biomedical major 2.22. 03 
UVI — Engagement — Course grade — Enrollment in second course — Biomedical major (CG-majority) 2.20 03 
UVI — Engagement — Course grade — Enrollment in second course — Biomedical major (CG-URM) .03 98 
UVI — Engagement — Course grade — Enrollment in second course — Biomedical major (FG-majority) —.47 64 
UVI — Engagement — Course grade — Enrollment in second course — Biomedical major (FG-URM) 2.25 03 
UVI X FG X URM — Engagement — Course grade — Enrollment in second course — Biomedical major 2.36 02 
Note. FG = first-generation college student status; URM = underrepresented racial/ethnic minority status. 


the value of course content (created by three writing assign- 
ments), students’ confidence in their ability to succeed can 
affect decisions about whether to persist, but that confidence is 
less important when the value of the domain is not emphasized. 

Moderated mediation analyses indicated that this intervention 
effect was sequential: within the UVI condition, more confident 
students were more likely to take the next biology course in the 
biology sequence, and enrollment in this course in turn led to 
long-term pursuit of a biomedical major. This finding is consistent 
with theorizing on how more proximal intervention effects may be 
“channeled” by the social system to affect longer term outcomes 
(Cohen, Garcia, & Goyer, 2017; Goyer et al., 2017). The UVI 
created a context that emphasized value, which led more confident 
students to complete another biology course that could help to 
fulfill biomedical major requirements, facilitating progress on an 
educational trajectory that increased the likelihood of persistence 
in a biomedical major. 


The Power of Linguistic Analysis 


Linguistic analysis permitted us to measure both personal 
focus and engagement. Importantly, we found that the direct 
intervention effects on long-term persistence were partially 
mediated by personal focus, an index of the degree to which 
students reflected, in their essays, on the personal relevance of 
biological topics in the introductory course. The UVI promoted 
reflection on the personal relevance of course material for all 
students, on average, and this increased personal focus in turn 
amplified the role of confidence in determining pursuit of a 
biomedical education. By helping students to appreciate the 
value of knowledge in the biomedical domain, the intervention 
seemed to catalyze the influence of students’ beliefs about 
whether they could succeed in the domain when deciding 
whether to persist. This finding is consistent with the positive 


expectancy-value interaction posited by expectancy-value the- 
ory (Eccles & Wigfield, 2002; Nagengast et al., 2011), which 
presumes that students who both expect to succeed in and 
personally value a domain are the most likely to pursue an 
education in that domain. However, this process may be a 
double-edged sword. In an introductory college course, students 
are making decisions about what field they want to pursue. An 
intervention that helps students focus on the personal value of 
biology may make confident students more likely to pursue 
biomedical majors, but for students who already have some 
doubts, focusing on the personal value of biology may lead 
them to decide they would rather pursue another field (e.g., a 
field in which they perceive both value and a high likelihood of 
success). 

Critically, we found that the process through which the UVI 
influenced long-term persistence was distinct from the process by 
which it promoted course grades for disadvantaged students. 
Whereas the direct effect on persistence was mediated by personal 
focus (and not by engagement), the direct effect on course grades 
for FG-URM students (and for all students, on average) was 
mediated by engagement with course content (and not by personal 
focus). These findings suggest that the UVI may initiate two 
distinct processes: (a) helping underperforming students to engage 
with course material, thereby improving performance and helping 
them believe they can pursue the domain; and (b) increasing 
reflection on the personal relevance of course material, thereby 
helping more confident students to see why pursuing that domain 
may be worthwhile. 


Implications for Intervention Science 


These findings underscore a critical principle of intervention 
science: interventions target specific outcomes through specific 
motivational processes (Harackiewicz & Priniski, 2018). Al- 
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A 140 123(88%) 123 (88%) 
AB-B 527 462 (88%) 450 (85%) 
BC-C 298 224(75%) 188 (63%) 
D 68 33(49%)  31(46%) 
F 6 0 (0% 117% 
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D FG-URMs 
Grade Earned N _ Enrollment Major 
A 4 3(75%)  4(100%) 
AB-B 21 17 (81%) 19 (90%) 
BC-C 29 22 (76%) 19 (66%) 
D 7 4 (57%) 4 (57%) 
F 3 0 (0% 1 (33% 


Figure 5. Descriptive statistics illustrating implications of indirect utility-value intervention (UVI) effects on 
persistence via course grade. Course grades were higher in the UVI condition than the control condition (panel 
A), particularly for first-generation—underrepresented racial/ethnic minority (FG-URM) students (panel B). 
Higher grades were in turn associated with higher levels of enrollment in the second course and majoring in a 
biomedical field, for the full sample (panel C) and for FG-URM students specifically (panel D). 


though the original study focused on improving performance, 
the current study focused on persistence and, thus, different 
processes were critical to this outcome. In the case of the UVI, 
we found that a single intervention influenced two separate 
processes (engagement with course material and reflection on 
personal relevance) and these processes led to effects on dis- 
tinct educational outcomes for different groups of students. Our 
findings suggest that researchers may benefit from identifying 
the most relevant outcome variables and considering whether an 
intervention may initiate multiple processes, which may in turn 
influence different academic outcomes. Furthermore, the cur- 
rent study builds upon prior work showing that linguistic anal- 
ysis can provide insight into the cognitive processes at play as 
students complete an intervention (e.g., Beigman Klebanov et 
al., 2017; Harackiewicz et al., 2016; Priniski et al., in press). 
Many interventions involve a writing component, and the words 
students use represent a rich and underutilized data source in 
intervention science (Harackiewicz & Priniski, 2018). 

Our results suggest that the UVI initiates multiple processes 
that culminate in increased performance in the course and 
increased persistence in the biomedical track in the long term. 
However, it is important to note that although the UVI has 
overall positive effects, some groups benefit more than others. 
First and foremost, the UVI improves course grades, for all 
students on average, for students with a history of poor perfor- 
mance, and for FG-URM students. The indirect effects observed 
in the current study suggest that these positive effects on grades 
may translate into long-term increases in persistence in the 


biomedical track. These results suggest that encouraging stu- 
dents to reflect on the utility value of topics in introductory 
STEM courses may be an effective way to promote persistence 
in the STEM pipeline for students who enter the course with a 
lower likelihood of succeeding, by helping them to obtain 
higher grades. 

The current study also indicates that there is a parallel process 
unfolding over time as a function of students’ confidence. The UVI 
creates a context in which students focus on the personal value of 
biology. In this context, students who are more confident in their 
ability to succeed in biology are more likely to persist in the 
biomedical track, which is consistent with expectancy-value the- 
ory. It is important to note that the simple effects of the interven- 
tion were not significant at high or low levels of confidence, 
suggesting that it is not the case that the UVI was good for some 
students and bad for others. Rather, the UVI amplified the effects 
of confidence, such that for students who received the UVI, 
confident students were more likely to persist in the biomedical 
track. One open question that will need to be answered in future 
studies is how this process would play out in introductory courses 
for nonmajors in which initial levels of confidence and value span 
a broader range. Students in the current study were all at least 
considering a major in the biomedical sciences. It is unclear 
whether the UVI would interact with confidence in the same way 
in a general education course in which most students are not 
intending to major in that field. 

In summary, the UVI is primarily a curricular intervention 
intended to promote engagement and performance in a particular 
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course. A growing body of research suggests that this intervention 
promotes engagement and performance in introductory courses 
and has especially strong effects for the students who need it most 
(Harackiewicz & Priniski, 2018; Walton & Wilson, 2018). In 
addition, the results of the current study suggest that the UVI is an 
intervention with the potential to increase persistence in the bio- 
medical sciences, in two ways. First, the UVI increases grades in 
important gateway courses for all students on average and FG- 
URM students in particular, and these higher grades result in a 
greater chance of persistence. Second, the UVI creates a context in 
which students focus on the personal value of biology, which (at 
least in an introductory course for biomedical majors) makes more 
confident students more likely to persist. Thus, the results of the 
current study help to clarify, on theoretical and practical levels, 
which groups are likely to benefit from the UVI in terms of 
performance and persistence. This provides important insights for 
educators and researchers who may be interested in implementing 
a UVLI, and provides a model for intervention scientists to inves- 
tigate the multiple processes and outcomes that may be impacted 
by a single intervention. 


Limitations 


Although this follow-up study allowed us to examine the influ- 
ence of the UVI on college biology students’ long-term educa- 
tional trajectories, there are several notable limitations. First, al- 
though the UVI promoted long-term persistence, our findings are 
limited to the context of the biomedical fields. It will be critical for 
future studies to investigate whether these effects generalize to 
other STEM fields. Second, although we provide evidence of 
intervention effects on students’ persistence in a biomedical major 
2 years postintervention, a next step for future studies would be to 
examine whether this influence can translate into even longer-term 
effects in the STEM pipeline (e.g., career choices). Finally, be- 
cause we found indirect, but not direct, effects of the intervention 
on long-term persistence in the biomedical track for FG-URM 
students, we cannot draw causal inferences about the potential of 
the UVI to increase diversity in the STEM fields. There are several 
possibilities that may explain our pattern of results. One possibility 
is that long-term direct effects for FG-URM students could not be 
detected because the positive effect through course grade was 
suppressed by negative effects through unmeasured mechanisms 
(Rucker, Preacher, Tormala, & Petty, 2011). Another possibility is 
that because these indirect effects are correlational, other external 
variables could underlie these long-lasting effects rather than the 
intervention. The results of our sensitivity analysis suggest that the 
indirect effects reported here were fairly robust to unmeasured 
confounding variables, but this possibility is nevertheless impor- 
tant to consider. Indirect effects are important to examine because 
they allow for investigation of the specific processes through 
which the proximal effects of an intervention may unfold over time 
to produce long-term impacts (Cohen, Garcia, Purdie-Vaughns, 
Apfel, & Brzustoski, 2009; Harackiewicz & Priniski, 2018; Hecht, 
Priniski, & Harackiewicz, 2019; MacKinnon, Krull, & Lockwood, 
2000; Rozek, Svoboda, Harackiewicz, Hulleman, & Hyde, 2017; 
Tibbetts et al., 2016). However, we interpret these indirect effects 
not as concrete evidence, but rather as a pattern that is consistent 
with our reasoning that the UVI may have lasting impacts on 


persistence in the biomedical track for underrepresented students 
by improving their performance in introductory courses. 


Conclusion 


A growing body of research indicates that well-timed, theoret- 
ically guided social-psychological interventions can affect impor- 
tant educational outcomes (Harackiewicz & Priniski, 2018; Walton 
& Wilson, 2018). The current study contributes to this research by 
showing that implementing a UVI in introductory STEM courses 
can affect long-term persistence in these fields. Furthermore, our 
results highlight the importance of thinking about interventions not 
as silver bullets that work through mysterious black boxes, but as 
precise strategies for addressing specific problems by targeting the 
relevant psychological processes (Harackiewicz & Priniski, 2018; 
Walton & Wilson, 2018). In summary, this study highlights the 
potential of a utility-value intervention to promote persistence in 
the STEM pipeline, and lends insight into the processes through 
which such long-term effects unfold over time. 
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